Acquiring depth information in a scene is one of the most fundamental and challenging problems in imaging. The estimation of the depth of an object in a scene finds many applications in areas such as robotics, gaming, gesture recognition, 3D visual entertainment and the like. In particular, depth information can bring added value to recorded images for Television and Cinema production. Indeed, information related to the depth of objects in a scene is useful for different purposes, such as visual effects and conversion of images from 2D-to-3D for 3D screening. Depth information is also useful for view synthesis at the viewing end, for example, to adapt the image content to the stereoscopic display used. Depth information can be delivered with the corresponding image data in the video stream, to the device rendering the images.
Techniques for acquiring depth information can generally be classified into two types: passive methods in which multiple cameras are used to record images of a scene from multiple different view-points and disparity estimation is performed using the acquired stereo images, and active methods in which depth is estimated based on active illumination of the scene and object reflection. For active methods Time-of-Flight (ToF) or structured light devices may be used. A ToF camera measures the phase shift of the reflected light relative to its modulated light source and calculates the distance between the camera and the observed objects. Each type of technique has its own advantages and drawbacks. Passive methods have been well-studied and only require video images. They are erroneous, however, in occluded regions (regions of the scene which are not visible from the selected viewpoints) and inaccurate in poor-textured regions (for example regions of uniform colour). Active methods, on the other hand, can provide accurate measurements in these critical areas but present other technology-specific limitations. For example, ToF devices produce dense depth maps but with very low resolution. On the other hand, structured light devices provide stable measurements with higher image resolution, but present holes in areas where the pattern cannot be detected, such as structured light shadows, absorptive, transparent and reflective material, as well as slanted surfaces.
Considerable attention has been dedicated to combining active and passive methods in the last few years, with the spread of affordable depth sensing devices. The proposed methods however are complex and time consuming. The present invention has been devised with the foregoing in mind.